Survivor Buddy Simulation for HRI

This paper extends the Survivor Buddy simulation in the Search and Rescue Game Environment (SARGE) through the addition of two-way audio communication and a specialized search and rescue multiplayer environment in order to support multi-person human-robot interaction studies. The multiplayer that was created through this project allows one person to be the survivor and another to control the robot. Through the addition of this new multiplayer environment, the ability to record an experiment from two viewpoints was added to SARGE and this will allow researchers to more easily create picture in picture videos. This project will affect the HRI community because this multiplayer environment can be used to experiment with new interaction techniques. As part of this project, two videos were developed, one with the Survivor Buddy always turning to face the victim's face and another with the Survivor Buddy always oriented towards the front of the robot. In order to expand these videos to include the Survivor Buddy view, a method was developed to create picture in picture videos. After completion of this phase, this multiplayer environment will be evaluated by researchers at Stanford to test for usability in future experiments. Introduction Rescue robots are created with functionality, not aesthetics, in mind. There are no extra parts that could potentially fail on these robots. The result is that these robots do not appear incredibly friendly; to see an example of a robot investigating a survivor, view Figure 1. The Survivor Buddy multi-player addition addresses this issue by allowing 2way audio and video communication with the survivors that are found in Urban Search and Rescue environments. The video communication will be a web cam view of the rescuer to be displayed on the screen and the feed from the robot’s camera and another camera affixed to the top of the screen. Initially the audio communication will be facilitated through the use of Skype running in the background. To better understand the impact of the Survivor Buddy, consider the story of the two Australian miners who were trapped in a mine collapse and requested a Foo Fighters album to keep them calm while they were waiting to be rescued. With this in mind, there are other forms of media that might be valuable to calm and comfort survivors. For example, in the case of a mine collapse it might be comforting to the survivors to know that their families are safe and they are the only ones that are in danger; on the other hand, victims of the September 11 th attacks would not be comforted by receiving more information about the attacks. The goal of this multiplayer environment is to test experiments to determine what aspects of the Survivor Buddy will be both useful and appropriate, and what may not fall into that category. This Figure 1: Example of a rescue robot investigating a survivor. multiplayer environment will be the first to allow multimedia experimentation as well as concurrent users in different roles. An environment such as this could easily be modified to run other HRI experiments and allow labs without the capital to obtain expensive robots to run tests that would otherwise have been out of reach. Related Work There are two main multi-player simulation environments other than SARGE that can be used for HRI experiments with manned robots: USARSim and a simulation created at Washington University in St. Louis. Neither of these environments currently support the type of multiplayer environment that was implemented in this project. USARSim consists mainly of environments that have been created in both the real world and in simulation for robot testing. One of these environments is pictured in Figure 2.1. SARGE also has environments that exist both in the real world and in simulation, but consists mainly of environments that are used for purposes other than search and rescue. Figure 2.2 shows the simulated test bed that is scheduled to be built at the University of South Florida as well as the iRobot Packbot that has been modeled for use in this environment. USARSim and SARGE both allow multiple operators to each operate a robot [1]. The simulation that was created by Josh Faust [2] at Washington University in St. Louis allows both human and robots to interact, but they interact as a team in order to complete Urban Search and Rescue tasks. As shown in Figure 2.3, this simulation is much less developed than the USARSim and the SARGE environments. This is very different from the environment that was created in that it focused on robots and humans being able to interact as a team, not as an experimental environment to test how the human reacts to the robot. Implementation SARGE was extended through the use of the Unity game engine to add this multiplayer environment for experimentation. For the Survivor Buddy multiplayer addition, 3 types of additions had to be made to the previously created scenes. The additions of the survivor, the rubble, and the Inuktun were made in Unity. In order for the multiplayer addition to work, spawn points were added for the survivor and the robot. Figure 2.1: USARSim environment Figure 2.2: SARGE environment Figure 2.3: Faust’s environment Figure 4.2: Bad HRI These points ensure that the players are placed in the correct places for interaction when the simulation is started. A script was added for the initial experiments that would allow the Survivor Buddy to either orient towards the front of the robot or towards the survivor’s face, which was shown to be important by Bethel [3]. In order to get feedback on the initial setup, two videos were created in iMovie to show the Survivor Buddy in each of its modes. Based on the feedback that was received, a few minor adjustments were made to the scene, such as darkening the background to make it more realistic. Pictures of the adjusted scene with the face-oriented Survivor Buddy and the robot-oriented Survivor Buddy are shown in Figures 4.1 and 4.2 respectively. Next, a picture in picture video was created using Final Cut Express HD in order to show the interaction from both the survivor’s and the robot’s perspectives, an example of this is shown in Figure 4.3. In order to create these videos, a camera was added to the Survivor Buddy model in the simulation to simulate the view that the operator would be able to see. The choice to switch to Final Cut was made due to the fact that iMovie cannot create picture in picture movies without a third-party plug-in being installed. Final Cut has quite a learning curve, but this is due to all the extra options that are afforded by this program. The multiplayer implementation was added by reviewing other multiplayer games that had been created through the use of Unity and figuring out how all the pieces fit together. Unity is a relatively new game engine and has really great tutorials for beginners, but not as many for advanced topics such as multiplayer. Jeff Craighead had already created a multiplayer environment, and he was able to offer some guidance. His code was a very valuable resource and the multiplayer code was based heavily on this. Instead of wanting the players to spawn in any of the spawn points, the players needed to be tied to a specific point. The multiplayer also needed to change from everyone playing the same robot to one person being a robot and the other being the survivor. Based on these changes to the multiplayer environment, it will be able to be used for experimentation. With minor changes, this environment should be able to be used for other experiments as well. For the initial experiments, Skype will have to be used for the 2-way audio communications due to developmental constraints in Unity. The ability for in game voice communication should be released in August and should be implemented by the end of Fall 2008. Figure 4.1: Good HRI Figure 4.3: Multiple viewpoints Figure 4.2: Bad HRI Evaluation The evaluation of this platform should take place in Spring 2009 through the use of contacts in the field of USAR robotics. Experts in the field will be contacted and asked to provide input on the system through the use of surveys. After the initial input is gathered and changes are made to update the system, experiments will begin. Stanford University will be in charge of running the experiments after the system has been tested for proof of concept. Conclusion Utilizing SARGE, a multiplayer environment was added for HRI experimentation. This addition will be included in future releases of SARGE, which can be freely downloaded from Sourceforge.net as long as Jeff Craighead is cited. Researchers should be able to use this environment to easily run experiments on the robots that are included in the release, or to add their own if they choose to. AcknowledgementsThe author would like to thank Dr. Robin Murphy for her encouragement andsupport throughout this project. Thanks also to Microsoft and CRA-W for their financialsupport. References[1] J. Craighead, “SARGE: A User-centric Robot Simulator and Training Environment,”presented at 2008 International Conference on Human-Robot Interaction,Amsterdam, Netherlands, 2008.[2] J. Faust, C. Simon, and W. D. Smart, “A Video Game-Based Mobile RobotSimulation,” in Proceedings of the IEEE/RSJ International Conference on Robotsand Systems (IROS 2006), 2006, pp. 3749-3754.[3] C. L. Bethel and R. R. Murphy, “Non-Facial/Non-Verbal Methods of AffectiveExpression as Applied to Robot-Assisted Victim Assessment,” in 2nd ACMSIGCHI/SIGART Conference on Human-Robot Interaction (HRI2007), 2007.